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Abstract 

This paper outlines a successful, stakeholder driven, continuous improvement process that 
is currently used at Estrella Mountain Community College. The process is designed to address the 
next step of what to do with institutional effectiveness data after it has been collected and reported. 
Many institutional effectiveness processes stop with the publishing of data. While this serves 
accountability purposes, it does little to ensure that data will be used for improvement purposes. 

To stimulate improvement around the College’s Core Indicators of Effectiveness, Estrella 
Mountain has designed a group interview process with the stakeholders responsible for delivering 
on different aspects on the Mission and Goals of the College. Using a set of interview questions, 
the College stakeholders identify standards for the Core Indicators of Effectiveness, develop 
strategies for improvement, and take ownership of these strategies. The interview approach has 
been used by the College for three years and has resulted in enhanced awareness of the 
institutional effectiveness process, increased participation by all stakeholders, and the development 
of concrete improvement strategies that are reviewed on an annual basis. 

Background and Purpose 

Estrella Mountain Community College is a two-year institution located in Avondale, 
Arizona. The college began offering classes in 1990 through local high schools and in the fall of 
1992 opened a permanent campus. 

Estrella Mountain offers both university transfer and multiple occupational programs to a 
student population of approximately 4,400 credit students. The College serves the greater 
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southwest portion of the Phoenix Metropolitan area. This service area includes a diverse 
population of approximately 200,000 residents. 

In the fall of 1995, Estrella Mountain developed its first Plan for Institutional Effectiveness. 
This Plan is patterned after the Core Indicators of Effectiveness for Community Colleges developed 
by the American Association of Community Colleges (AACC) Community College Roundtable in 
1993 (Alfred, 1994). Like the AACC Core Indicators of Effectiveness model, Estrella Mountain’s 
institutional effectiveness program is directly linked to the achievement of the institution’s 
Mission. The Plan includes 22 Core Indicators of Effectiveness linked to seven College Mission 
and Goal areas. These mission areas include: Student Success, Transfer Education, General 
Education, Developmental Education, Workforce Development, Student Support Services and 
Community Education. The Estrella Mountain model of Institutional Effectiveness is included as 
Appendix A. 

Estrella Mountain has also developed a separate but related effort for the assessment of 
student learning. While the purpose of the Plan for Institutional Effectiveness is to evaluate if 
students and the community are getting what they want from the College, the purpose of the 
Academic Achievement Plan is to assess if students are learning. Hence, the Plan for Institutional 
Effectiveness addresses core indicators related to goal attainment, transfer success, job placement, 
student satisfaction, access, and other related areas. The Student Academic Achievement Plan 
addresses if students are progressing in General Education abilities such as critical thinking and 
communication as well as program specific competencies. 

This paper addresses a continuous improvement process related to the core indicators of 
effectiveness found in the College’s Plan for Institutional Effectiveness. It does not address the 
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processes that are used by the student academic achievement effort to improve student-learning 
outcomes (this process takes the form of faculty dialogues). 

Soon after the adoption of the Plan for Institutional Effectiveness, the College’s planning 
and research staff collected data for most of the College’s core indicators and related measures. 
The first use of these data were published and analyzed in the Estrella Mountain 1996 Self-Study 
conducted for the North Central Association of Colleges and Schools (NCA). The data were very 
useful in the self-study process and helped to identify some of the College’s strengths, weaknesses 
and challenge areas. However, the self-study process did less to identify improvement strategies 
around these core indicators. Also, since accreditation occurs only once every five to ten years, it 
does not provide a regular forum to review core indicator of effectiveness results. It is for this 
reason that the College designed and implemented a stakeholder driven continuous improvement 
system around its core indicators of effectiveness. 

Literature Review on Improvement Processes 

While much is published on the development of effectiveness models, data collection 
strategies, state accountability mandates, and the development of “Report Cards”, there are fewer 
examples of how colleges and universities use institutional effectiveness data to stimulate 
improvement. 

The recent movement among some community colleges and universities toward quality- 
based systems should provide a greater depth of research related to the development of 
continuous improvement systems in the educational environment. Several state quality award 
programs and the Malcolm Baldrige National Quality Award Program mandate the use of 
process improvement. Even accrediting bodies, such as NCA, are beginning to adopt continuous 
improvement methodology versus episodic site visits. As of April 2000, the North Central 
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Association of Colleges began a pilot program called the Academic Quality Improvement Project 
(AQIP). The AQIP process requires institutions to articulate their goals in measurable terms and 
develop systems to improve the institution’s overall effectiveness on a continuous basis. In this 
system, colleges are required to provide written updates on their progress to NCA on a 
continuous basis. The use of site visits (once every seven years) is still used, but the visit is 
focused on evaluating the quality of the improvement process (AQIP, 2000). The continuous 
improvement process outlined in this paper can be integrated into various quality programs like 
the ones mentioned. 

The Estrella Mountain Continuous Improvement Interview Process 

What follows is an outline of the Estrella Mountain Continuous Improvement Interview 
process. 

Identify the Stakeholders 

The first and most important step in the process is to identify individuals who have a 
“stake” in each of the College's Mission Goals (mission goals may be referred to as “purposes” at 
other colleges). These individuals are called “stakeholders.” The list should include both internal 
stakeholders (employees responsible for delivering on certain aspects of the college mission) and 
external (students, community members, business owners, etc.). The identification process is 
accomplished by the Estrella Mountain Institutional Effectiveness Steering Team (already a cross 
functional team of employee stakeholders). These stakeholders represent the potential participants 
in the continuous improvement process. 

After the master list of stakeholders has been identified, a Lead Stakeholder is assigned to 
each Mission Goal. The Lead Stakeholder is usually a volunteer staff member that has a job 
function that is directly related to one of the Mission Goals. The Lead Stakeholder serves as 



champion for the goal they are assigned. This staff or faculty member is responsible for inviting 
other stakeholders to attend one of the annual continuous improvement interviews and following 
up on the progress of improvement strategies that are identified during the interview. Using a 
Lead Stakeholder increases stakeholder ownership and positively impacts attendance and 
participation in the improvement strategies. The Lead Stakeholder role could be administered by 
one of the research staff, but this is less effective due to decreased ownership in the process. 
Scheduling Improvement Sessions 

Rather than conducting a single session covering all Estrella Mountain’s Mission Goals, the 
sessions have been divided into separate goal areas. At Estrella Mountain, the improvement 
sessions have been broken down into: General and Transfer Education, Developmental Education, 
Workforce Development, Student Support Services, Community Education, and Student Success. 
These groupings are clustered around the College Mission and Goals statement. The Estrella 
Mountain Mission Goals statement is included as Appendix B. 

The first institutional effectiveness stakeholder interviews were conducted during the 
spring 1998 semester. The sessions have continued each spring semester since then. Starting in 
the 2000-2001 school year, interviews will be scheduled during the fall to spread out the interview 
schedule. 

The Interview 

Once dates have been identified, interviews are conducted with each of the Mission Goal 
stakeholder groups. Each group generally includes 25-30 stakeholders. The interview requires 
approximately one and a half to two hours to conduct. The following outlines the major 
components of the interview. The actual interview questions are included as Appendix C. 
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Review of the Institutional Effectiveness Process (15 minutes) 

This portion of the interview includes an overview of the Institutional Effectiveness model and the purpose 
for the interview. This part of the process is most important for stakeholders that are new to the process. 

Review of Previous Year’s Improvement Strategies (10 Minutes) 

The status of the previous year’s improvement strategies is reviewed. Participants are asked to evaluate which 
strategies worked well and which ones didn’t work well or were not implemented. 

Overview of Key Findings for the Core Indicators of Effectiveness (25 minutes) 

Prior to the interviews, all stakeholders are provided both an executive summary (3 pages maximum) of key 
findings related to the core indicator results as well as detailed information for all core indicator measures. 
During the interviews, a PowerPoint presentation of the key Core Indicator results is conducted and 
participants are allowed time to review and re-read the Executive Summary. Questions concerning the Core 
Indicator data are also addressed at this time. 

Identification of Strengths and Improvement Opportunities (25 minutes) 

Stakeholders are asked to identify both strengths and opportunities for improvement based on the review of 
the Core Indicator results. Good facilitation skills are required during this part of the interview. 

Identification of Improvement Strategies and Volunteers (25 minutes) 

The participants are asked to review the opportunities for improvement and to identify strategies that could be 
employed to improve on the Core Indicator results. If more than a few strategies are identified, a 
prioritization technique is used to identify what the group believes are the most critical strategies to be 
implemented. Volunteers are identified who will “champion” each strategy. Both prioritization and 
identifying champions are critical components in the process. 

Review and Setting of Core Indicator Standards (10 minutes) 

One goal of the Core Indicator of Effectiveness process is to identify standards for all Core Indicator 
measures. A standard represents the minimum achievement level for Core Indicator Measures and often is set 
using national or district level data. For example, standards can be set for items such as transfer rates, 
retention, job placement, etc. Standards are more difficult to set when benchmarks are not easily identified. 
The use of standards helps to identify areas that should be addressed via additional research or continuous 
improvement strategies. Each year, the stakeholders are asked to review the usefulness of the current 
standards as well as set standards for new measures. If a standard can’t be agreed to, the stakeholders are 
asked to identify the information they need to be able to set a standard. 

Modifications to Core Indicators and Measures (10 minutes) 

The Core Indicators of Effectiveness and Measures are reviewed by the stakeholder group. Most of the time 
the Core Indicators and Measures are reaffirmed. Occasionally there will be recommendations to modify 
Core Indicators and or Measures due to changes in college programs. The changes to the Core Indicators are 
implemented and brought back to the following year’s continuous improvement interview for adoption. 

Plus/Delta (5 minutes) 

All interviews end with an evaluation of the continuous improvement interview process. Participants are 
asked to identify what they liked best about the process as well as what can be done to improve the process. 
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Lessons Learned 



The stakeholder interview process has resulted in several lessons learned. The following 
suggestions provide guidance in building a good continuous improvement process: 

• Stakeholder ownership is critical. Stakeholders should to be included in both the 
development and implementation of the improvement system. 

• Obtain senior level administrative support. College deans should actively participate in the 
improvement process and the college president should endorse the process. 

• Always review the purpose of the process, particularly if you have negative data to report. 
The process should be focused on “continuous improvement” not “assignment of blame”. 
This process cannot work if employees fear that the data will be used against them. 

• During the interview, use the executive summary to its fullest. Keep the findings simple 
and limit executive summaries to two to three pages. Do provide detailed information, but 
don’t spend interview time reviewing data “line by line”. 

• Divide and conquer. Start with one part of your institutional effectiveness program and 
pilot the effort. Provide enough time between interviews to allow for adequate preparation. 
If possible, spread the continuous improvement interviews over an entire semester or 
academic year. This helps reduce the chance of “bum out” by stakeholders who are part of 
multiple stakeholder groups. 

• Always review progress on the previous year’s improvement strategies. This documents 
the value of the process (useful for accreditation) and also provides motivation to current 
stakeholders when they see that their strategies have been accomplished. Even when 
strategies haven’t been implemented, many employees are motivated by peer pressure to 
follow up on these strategies during the next year. 
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• When new improvement strategies are identified, always identify a group of volunteer 
employees to work on the strategy. It’s also important to prioritize improvement strategies 
if many are identified. 

• Have the Key Stakeholder request progress reports from the volunteer strategy leaders 
three to four months after the interview. This helps to ensure that progress is made toward 
the identified strategies. 

• Plan for follow up time with the stakeholder groups. Often the group interview will spawn 
improvement strategies that require a partnership with the research office. 

Future Enhancements to the Process 

Over the next year, The College is considering adding the following to the Stakeholder 
Continuous Improvement Interview process. 

• External stakeholders (students and community members) will be included in the 
continuous improvement interview process during the 2000-2001 school year. 

• WEB-based tools will be used to share core indicator results, exchange ideas outside of the 
scheduled interviews, and track the progress of improvement strategies. The College is 
currently adapting the Blackboard course management tool to accomplish this task. While 
the tool is primarily designed for course delivery, it can be adapted for use in committee 
work and tasks forces. More information is available on Blackboard at 
www.blackboard.com. 
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Appendix B 



Estrella A Mountain A Community A College 



Estrella Mountain Community College provides opportunities for 
our students to achieve success in educational and personal goals. 

We are an institution of higher education serving the West Valley 
communities through: 



General Education 

To provide the knowledge and abilities that enable students to 
achieve academic and personal goals. 

Transfer Education 

To provide quality transfer courses and programs that enable 
students to achieve success at four-year institutions. 

Developmental Education 

To provide quality developmental courses and programs that 
prepare students for educational and personal success. 

Workforce Development 

To provide specialized quality training, courses and services that 
meet the needs of businesses and individuals. 

Student Support Services 

To provide quality services and resources that meet the needs of 
students and support learning. 

Community Education 

To provide a wide variety of opportunities that meet the needs of 
life-long learners. 



MISSION 



• General Education 

• Transfer Education 

• Developmental Education 



• Workforce Development 

• Community Education 

• Student Support Services 



GOALS 



Appendix C 

Estrella Mountain Community College 
Continuous Improvement Interview Questions 
Spring 2000 



After the core indicator of effectiveness data are presented and reviewed, the following interview 
questions are asked of the key stakeholders. 



1 . Based on the core indicator results for this Mission Goal, what strengths do you see? 



2. Based on the core indicator results for this Mission Goal, what opportunities for 
improvement do you see? 



3. What improvements strategies worked well last year, which ones didn’t work as well or 
weren’t implemented? 

4. What are some actions (improvement strategies) that you might consider implementing in 
your area to positively improve future core indicator results for this Mission Goal for the 
coming year? (This list will be prioritized.) 



5. For those Core Indicators and Measures that do not have identified standards, what should 
they be? What additional information would you like to have to set more ideal standards? 
For those Core Indicators and Measures that do have standards, are they still appropriate? 



6. Are there any modifications that should be made to the Core Indicators? What are they? 
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